Characterizing Traffic in Widely-Deployed DHT

A Peer to Peer (P2P) network is one of the most popular forms of distributed resource sharing. The idea behind P2P networks is that several users (peers), distributed globally, share different resources among themselves without the presence of a central management authority. Examples of the resources are files, CPU cycles and network bandwidth. The most common use of P2P networks is to share files. P2P networks are divided into 2 major types: unstructured and structured. In structured networks, a distributed algorithm ensures that all the peers adhere to a particular structure. This is similar to an IP network, in which all the routers collectively form a tree structure. The presence of a structure ensures that 2 peers can contact each other and share resources by simply following the path dictated by the structure. As a result, if a resource exists in the network then it will be found. In an unstructured network, on the other hand, no such structure exists. Peers join the network by simply contacting random peers. Peers search for resources by flooding the network. As a result, even if a given resource exists in the network, there is no guarantee that it will be found. The most popular form of structured networks is Distributed Hashtables (DHTs). Just like other P2P systems, DHTs are mostly used for filesharing. A DHT imposes a particular structure by assigning unique identifiers to peers, which define their position in the network. The main challenge is how to find location of files in the network. This is achieved by cooperation between peers contributing files (publishers) and peers searching for files (searchers). A DHT peer publishes a file by creating a hash of the file. Next, it finds a peer whose ID is closest to the file hash in the ID space. The metric for the closeness ranges from arithmetic difference to pattern matching. Finally, the publisher sends the file meta-data, such as size, name and type to the peer closest to the file ID. This meta-data will be used for future file searches. A DHT peer searches for a file by finding a peer closest to the file hash and retrieving the file meta-data. Finally, by using the file meta-data, the peer searching for file contacts the file publisher and downloads the file. The Above discussion indicates that the process of publishing a file as well as searching a file requires finding a closest node to the file ID. DHT peers achieve this by maintaining id, ip and port number of neighboring peers in their routing tables. Each peer finds the closest peer to a particular file id by choosing and querying the closest peer from its own routing table. The chosen peer repeats the same process and finds the closest node from its own routing table for the queried file ID and informs the request originator. The request originator then contacts the newly discovered peer. This process continues until the closest peer to the file ID is found. DHTs have been an active area of research since 2001. The idea of DHTs was introduced by near simultaneous introduction of 4 core DHTs: CAN [1], Chord [2], Pastry [3] and Tapestry [4]. Later studies, such as OpenDHT [15], Accordion [16] and Kademlia [7], improved DHT design for real world deployment. However, none of these DHTs have been widely deployed in a real world setting, until recently. As a result, real world properties of DHTs are not well understood. Some earlier studies, such as Performance vs. Cost Framework [17] and Impact of DHT routing [18], analyzed different characteristics of existing DHTs. However, these studies were based on simulations, which cannot account for real world conditions. A real world DHT can be characterized from 2 different angles, as follows: